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Executive  Sinnmary 


Purpose 


Background 


Results  in  Brief 


Over  33  million  beneficiaries — retirees,  survivors,  and  dependents — 
received  about  $175  billion  in  Social  Security  benefits  during  fiscal  year 
1986,  These  beneficiaries  rely  on  the  Social  Security  Administration 
(ssA)  to  calculate  and  pay  the  correct  benefits,  ssa  measures  how  accu¬ 
rately  it  calculates  and  pays  benefits  by  reviewing  an  annual  sample  of 
payment  cases.  Because  accuracy  rates  an'  us('d  as  barometers  of  how 
well  SSA  carries  out  its  fiduciary  responsibilitit's  to  beneficiaries  and  the 
trust  funds,  gao  sought  to  determine  the  adequacy  of  ssa's  measurement 
procedures  and  the  accuracy  of  the  rates  repoited  annually  for  the 
Retirement  and  Survivors  Insurance  ( Ksi )  program,  ssa's  largest  pro¬ 
gram,  GAO  also  measured  the  effect  of  errors  on  betu'ficiaries. 


ssa’s  10  regional  assessment  offices  annually  review  a  sample  of  about 
2,000  payment  cases  to  determine  accuracy  for  the  ksi  program.  This 
review  is  an  imperfect  barometer  in  that  it  is  not  designed  to  identify 
with  certainty  the  many  specific  causes  of  errors.  Other  ssa  samples  are 
designed  to  analyze  causes  of  errors  and  monitor  changes  in  processing 
accuracy,  but  cannot  be  used  to  project  overall  ksi  payment  accuracy 
because  they  do  not  include  certain  cases,  ssa  uses  the  results  of  its 
annual  payment  accuracy  sample  to  compare  overall  trends  in  Rsi  pro¬ 
gram  accuracy  over  time. 

In  1985,  SSA  began  reporting  an  rsi  payment  accuracy  rate  to  the  House 
Appropriations  Committee,  which  expressed  concern  about  the  impact 
staff  reductions  might  have  on  payment  accuracy.  s.sa  reported  that  its 
payment  accuracy  rate,  expressed  as  a  perct'ntage  of  total  dollars  paid, 
fluctuated  between  99.5  and  99.6  percent  for  fiscal  years  1981-86. 

GAO  evaluated  ssa’s  processes  for  measuring  pav'ment  accuracy,  using 
the  findings  of  ssa’s  quality  review  analysts  for  1981-86  to  verify  the 
reported  accuracy  rates,  gao  also  statistically  compared  the  perform¬ 
ances  of  the  SSA  regional  assessment  offici's  to  d('t('rmine  whether  varia¬ 
tions  in  the  error  rate  detected  might  he  due  to  something  other  than 
chance.  P'inally,  g.ao  used  ssa's  lf)84  payment  sampk-  to  calculate  the 
effect  of  errors  on  beneficiaries 


ssa's  sampling  methodology  and  j)rocesses  for  measuring  ('i  rors  and 
determining  annual  payment  accuracy  rates  are  ad('(iuat('.  lIow('ver.  not 
all  errors  detected  are  includi'd  when  accurac\  rates  are  calculated 
hecau.se  of  how  .s.sa  int<'rpr('ts  errors.  ( 'on.se()nenl  1\ ,  gao  helii'vc's  actual 
error  rates  are  about  twice  what  ss\  (  aU  ulatc's  When  ssa  doi's  not 
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Principal  Findings 

Incidence  of  Error  Not 
Reported 


include  all  errors  in  calculating  and  reporting  accuracy  rates,  it  over¬ 
states  program  performance  and  can  mislead  managers  about  the  extent 
to  which  potential  improvements  might  be  realized. 

When  SSA  reported  its  annual  payment  accuracy  rates  for  198 1-8(5  to  the 
House  Appropriations  Committee,  it  reported  an  accuracy  rate  ba.sed 
only  on  total  dollars  paid,  not  on  the  incidence  of  cases  in  error,  which 
was  less  favorable.  Further,  ssa’s  reported  accuracy  rate  did  not  include 
underpayments  and  certain  overpayments,  which  .ssa  believed  should 
not  be  reported  as  errors,  nor  did  the  rate  differentiate  between  errors 
caused  by  ssa  and  those  caused  by  others. 

When  GAO  counted  all  detected  errors  for  198t),  the  case  accuracy  rate 
was  about  87.3  percent  (compared  with  s.sa's  94.1  percent),  which 
means  that  about  4.2  million  persons  were  overpaid  or  underpaid  bene¬ 
fits  totalling  $1.1  billion.  Recalculating  error  rates  for  1981 -8(5  to  include 
all  errors  ssa  detected  did  not  change  the  overall  downward  trend  in 
errors. 

SSA  payment  accuracy  rates  measure  overall  Ksi  program  accuracy,  but 
do  not  measure  how  errors  affect  beneficiaries.  When  gao  calculated  the 
total  impact  of  all  errors  on  individual  cases  over  time,  it  found  almost 
two-thirds  of  errors  were  underpayments:  because  of  the  nature  of  the 
errors,  most  would  not  have  been  likely  to  be  detected  by  routine  ssa 
processes. 


For  1981-8(5,  ssa  repoiled  an  average  99.5  i)erc('nt  payment  accuracy 
rate  biused  on  t(»tal  program  dollars  and  not  the  average  92.1  percent 
ca.se  accuracy  rate  ba.sed  on  .ss.\  criteria.  'I’his  gave  an  overly  favorable 
im{)ression  of  payiiKMit  accuracy.  .Mthough  both  rates  are  apf)ropriate 
measures  calculated  by  ssa.  the  ca.se  accuracy  rat('  is  :i  better  indicator 
of  the  incidence  of  error;  t  he  dollar  accuracy  rate  is  a  iM'tti'r  indicator  of 
the  overall  im[)act  of  errors  on  |)rogram  finances.  .s,sa  said  that  outside 
SSA.  it  rejMHied  the  aicuracy  rate  based  only  on  total  dollars  paid 
because,  in  tlu*  pa.st.  it  had  rei>orted  this  figun'  to  those  iiujuiring  about 
overall  payni<*nl  a<'curacy.  (,'sc<>  p.  i;{.) 
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Some  Errors  Are  Not 
Reported 


In  reporting  dollar  accuracy  rates,  ssa  understates  the  extent  of  errors 
because  its  criteria  for  errors  do  not  include  (1)  underpayment  errors. 

(2)  errors  of  $  1  to  $5  per  month,  and  (3)  errors  barred  from  correetion 
because  they  occurred  over  4  years  ago.  s.sa  has  consistently  excluded 
these  categories  from  its  overall  accuracy  rates:  the  first,  because  it  does 
not  represent  dollars  incorrectly  paid,  and  ssa  calculates  a  separate 
underpayment  accuracy  rate  but  does  not  report  this  outside  ssa;  the 
second,  because  the  amount  of  the  error  is  relatively  insignificant;  and 
the  third,  because  payments  that  ssa  can  no  longer  correct  are,  accord¬ 
ing  to  administrative  procedures,  no  longer  incorrect  payments.  To  fully 
disclose  all  errors  occurring,  gao  believes  all  errors  detected  should  be 
included  when  calculating  and  reporting  accuracy  rates.  (See  pp.  12-14.) 

Exclusion  of  errors  detected  significantly  affects  the  number  of  cases 
reported  in  error.  When  gao  included  all  detected  errors,  the  case  error 
rate  averaged  about  15.0  percent  or  about  twice  what  ssa  calculated 
using  its  criteria.  The  case  error  rate  steadily  decreased  from  19.7  per¬ 
cent  in  1982  to  1 1.5  percent  in  1985;  this  was  primarily  due  to  fewer 
cases  with  recent  earnings  being  underpaid  because  ssa  computed  their 
benefit  increases  more  quickly.  The  error  rate  increased  in  1986  to  12.7 
percent,  but  such  increase  could  be  due  to  the  chance  inherent  in  ran¬ 
dom  sampling.  Over  the  6-year  period,  an  average  of  about  4.0  million 
cases,  representing  an  average  of  about  4.7  million  persons,  were 
affected  by  errors  each  month. 

Although  ssa  determines  whether  errors  it  detects  were  caused  by  it  or 
others,  ssa  has  not  reported  this  outside  s,sa.  P'or  1981-86.  the  incidence 
of  cases  reviewed  found  to  be  in  error  and  attributed  to  ssa  varied  from 
14.4  percent  in  1982  to  6.2  percent  in  1,985.  It  was  7.0  percent  in  1986,  a 
statistically  insignificant  increase.  Reporting  the  errors  caused  by  ssa 
would  provide  better  compari.sons  of  ss.\’s  performance  with  earlier 
years.  (See  pp.  14-21.  ) 


Some  Errors  May  Be 
Undetected 


Because  two  regional  assessment  offices  may  not  be  detecting  all  ('rrors 
in  sampled  ca.ses,  KSi  errors  may  be  higher  than  indicated  in  this  report. 
(iAO  combined  5  years  of  sample  data  and  compared  and  analyzed  the 
error  detection  rates  of  the  10  regional  assessment  offices,  (iAo  found  a 
disparity  in  detection  rates  for  the  (Oiicago  and  I)en\  er  regional  offices 
that  could  not  be  attributed  to  chance.  This  disjiarity  was  not  found  by 
the  SSA  headquarters  group,  which  sampU's  and  validates  I  he  i  t'gional 
assessments  hut  has  not  doni'  a  disparit\  anal\ sis.  I-’rom  the  a\  adahlc 
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Recommendations 


(;a()  rocommends  that  the  Secretary  of  Health  and  Human  Services  (ims) 
direct  the  Commissioner  of  Social  Security  to  make  a  more  complete  dis¬ 
closure  of  Ksi  payment  errors.  Specifically,  the  case  error  rate  and  the 
dollar  error  rate  should  be  reported.  In  addition,  the  reported  rates 
should  (1 )  include  all  cases  with  detected  errors  and  (2)  indicate  the 
incidence  of  errors  attributed  to  .ssa,  .s.sa  should  use  such  error  rates  to 
better  assess  program  performance  and  to  decide  the  extent  of  improve¬ 
ment  it  deems  attainable. 

(;ao  al.so  recommends  that  the  vSecretary  direct  the  Commissiont'r  to 
improve  monitoring  of  regional  assessment  reviews  and  determine  the 
following;  the  extent  to  which  future  regional  reviews  are  not  detecting 
errors,  any  actions  needed  to  ensure  all  errors  are  detected,  and  the 
effect  on  payment  accuracy  rates  of  any  errors  not  detecti'd. 


Agency  Comments 


ims  gi'iierally  agreed  with  the  thrust  of  the  report,  mis  said  it  would  ( 1  ) 
review  overall  procedures  for  reporting  payment  accui  acs'  data  and 
reexamine  its  definitions  and  practices,  (2)  report  ca.s(>  as  well  as  dollar 
accuracy  in  the  future  for  both  overpayments  and  underjniyments,  and 
td)  report  the  extent  of  error  attrihuted  to  ss\.  mis  also  said  it  planni'd 
to  determine  the  reasons  for  the  disjiaritii's  in  ri'gional  dc'tt'ction  l  atcs 
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Chapter  1 


Introduction 


Many  Factors  Can 
Affect  Benefits 


S  V-  ',v. 


Social  Security  beneficiaries — retirees,  survivors,  and  dependents — rely 
on  the  Social  Security  Administration  (,ssa)  to  accurately  calculate  their 
benefits.  To  measure  the  accuracy  of  its  payments,  ssa  samples  pay¬ 
ments  made  and  calculates  an  accuracy  rate,  which  is  reported  to  the 
Congress  and  others  as  indicative  of  how  well  ssa  does  its  job. 


The  importance  of  SSa’s  payment  accuracy  has  increased  as  staff  cuts 
and  other  efficiency  initiatives  are  considered  and  implemented.  In 
1986,  the  House  Committee  on  Appropriations  directed  the  Commis¬ 
sioner  of  Social  Security  to  report  to  the  Committee  on  the  accuracy  of 
payments,  along  with  other  performance  indicators.’  The  Committee 
stated  that  these  data  would  be  particularly  important  in  the  future 
because  of  planned  changes  in  ssa’s  staffing,  field  structure,  data 
processing,  and  telecommunications. 


This  report  (1)  assesses  the  validity  of  ssA-reported  accuracy  rates  for 
the  Retirement  and  Survivors  Insurance  (rsi)  program,  ssa's  largest  pro¬ 
gram,  and  (2)  measures  how  inaccurate  payments  affect  beneficiaries. 


Measured  by  the  number  of  beneficiaries  and  the  dollar  amount  of  pay¬ 
ments,  RSI  is  the  largest  program  that  ssa  administers.  In  fiscal  year 
1986,  SSA  paid  over  33  million  beneficiaries  about  $175  billion  in  bene¬ 
fits.  RSI  benefits  are  paid  to  partially  replace  earnings  lost  when  workers 
retire  or  die.  ssa  must  consider  many  factors  related  to  the  worker  when 
determining  eligibility  for  and  the  amount  of  the  initial  benefit,  for 
example:  amount  of  lifetime  earnings,  time  period  w'orked,  date  of  birth, 
family  relationship  to  others  (such  as  spouse  and  children),  entitlement 
to  other  benefits,  and  military  service.  After  the  benefit  payments  begin, 
events — postretirement  earnings,  marriage,  death,  and  the  birth  of  or 
changes  in  the  age  of  children — could  affect  subsequent  benefits,  ssa 
carmot  totally  control  the  accuracy  of  initial  or  subsequent  Rsi  benefits 
because  it  depends  on  beneficiaries  and  others,  such  as  employers,  to 
report  accurate  and  timely  information.  Nevertheless,  precise  payment 
accuracy  measurements  are  important  because  they  can  disclose  trends 
in  accuracy  over  time  and  can  help  identify  the  types  of  payment  ('rrors 
occurring. 


'  For  fiscal  year  1!ISI>.  S,S..\  pnividcd  (|iiai1ciiy  rc|Hirls  nii  sen  irc  dal  a  dins  tl\  In  Ihc  (  oniircss  |•■||| 
fist  ai  yt*ar  lfW7  ami  ai  I  ho  roqiiost  of  tho  I  imisr  and  S'lialr  (  niiiininrfs  <  in  .Vpi'nipnai  i*  nis  aw 
ov.ihiatin^J  sonioo  data  ocmipilod  Ity  SS.\  ami  pn>\  ulin^  n“|>nris  in  t  iio  ( 'nnimii  tf«’s  nn  t  lu‘  id  1<‘<  ts  n! 
staff  n*(luotions  on  mtvuo 
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How  SSA  Measures 
Payment  Accuracy 


The  Office  of  Assessment  within  ssa  reviews,  evaluates,  and  measures 
the  accuracy  of  Social  Security  programs,  including  the  Ksi  program,  and 
makes  recommendations  for  corrective  action.  For  the  Rsi  program,  the 
Office  of  Assessment  uses  three  routine  studies  that  address  different 
program  quality  aspects:  f'irst,  the  Awards  and  Disallowances  Study, 
which  samples  actions  weekly  and  reviews  about  (50, ()()()  ca.ses  annually, 
measures  compliance  with  policies  and  procedures  for  approving  initial 
claims.  Second,  the  Postadjudicative  Study,  which  also  samples  actions 
weekly  and  reviews  about  30,000  cases  annually,  measures  the  accuracy 
of  actions  taken  after  a  person  becomes  eligible  for  benefits.  And  third, 
the  Payment  Accuracy  Study,  which  annually  samples  and  reviews 
about  2,()00  ctises.  estimates  the  overall  accuracy  of  Social  Security  pay¬ 
ments  (benefit  payments  to  people  residing  outside  the  country  are  not 
sampled  as  part  of  this  study). 

The  Payment  Accuracy  Study  sample  includes  both  cases  with  actions 
during  the  year  (such  as  initial  claims  and  po.st-adjtidicative  actions)  and 
cases  for  which  no  actions  hav(>  been  taken.  It  is.  therefore,  the  only 
study  from  which  an  overall  annual  accuracy  rate  can  b('  projected  on- 
sequently.  its  value  is  primarily  that  of  an  overall  accuracy  indicaio’ . 
This  study  also  collects  information  about  the  causes  of  errors,  but  the 
sample  size  is  too  .small  for  statistically  valid  conclusions  to  be  drawn. 
More  detailed  data  on  the  causes  of  payment  erroi's  ai'e  derivt'd  from  the 
other  two  studies. 

The  Payment  Accuracy  Study  currently  samples  .lanuary  benefit  pay¬ 
ments,  and  the  accuracy  of  that  month's  check  is  reviewi'd.  samph'd 
check  may  include  benefit  i)ayment.s  for  more  than  one  beneficiary 
because  multiple  persons  may  be  entitled  to  Ixmefits  under  a  single 
Social  Security  account  number.  The  address  of  the  sanqjled  beneficiar\ 
determines  which  of  10  regional  assessment  offices  will  be  assigiK'd  the 
case  for  review.  At  a  regional  a.ssessment  office',  one  or  more'  quality 
review  analysts  will  determine'  the'  eorre'e'tne'.ss  of  the'  case  by  i-e'\  iewing 
the-  e  laims  feeleie'r.  maintaine'd  and  submitteel  by  one'  of  six  jH-ogram  scr- 
vie'e  cente'rs  (I’se  ).  If  ivqeiire'el.  an  analyst  will  inte'rvie'w  the'  be'ncficiarv 
anel  otlu'i's.  stu  b  as  e'luploye'rs,  ;mel  gatbe'r  eorrolieefat ing  e\  ide'iice'  IVom 
diffe're'iit  semre  e's.  sue  b  as  state'  Bure-aus  of  \  ital  Statistie  s. 

For  eae  b  e-ase*.  an  analyst  tbe'ii  e  etele's  the  finelings.  sue  b  as  t  be  dollai' 
amount  ot  tbe*  payme'nl  e'rror  ss\  has  se't  efite'i  ia  tin'  de’termining  an 
erreer  eliiriug  a  ca.se'  re'vie'W  and  for  determining  one  for  ri'porling  pur 
[lose’s.  For  inslane  e',  tlu'  e  rite'i  ion  foi'  an  e'rror  ebn  ing  a  e  ase-  rc\  lew  m  i  be 
feilleiwing:  t  be- sample'ei  memtbly  be'iu’fit  amoimt  eliflei  s  from  w  liai  il 
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Objectives,  Scope,  and 
Methodology 


Chapter  1 
Introduction 


would  have  been  had  the  case  been  handled  correctly  from  the  begin¬ 
ning.  Detected  instances  of  these  differences  are  coded  and  entered  into 
the  data  base.  The  Office  of  Assessment  then  applies  different  criteria 
(for  example,  the  dollar  amount  of  the  error  must  be  $5  or  more)  to 
determine  the  errors  in  the  data  base  that  are  “reportable"  (see  p.l2  ). 

Other  findings  the  analyst  codes  are  the  cause  (139  specific  types)  and 
the  responsible  party  (SSA,  beneficiary,  or  other);  the  analyst  enters 
these  findings  by  computer  terminal  into  the  payment  accuracy  data 
base  maintained  at  ssa  headquarters.  These  findings  may  be  re-reviewed 
through  a  paperwork  evaluation  at  ssa  headquarters,  where  staff  use  a 
subsample  to  validate  the  findings  of  the  regional  case  reviews.  From 
the  data  base,  the  Office  of  Assessment  produces  payment  accuracy  sta¬ 
tistics  that  are  expressed  as  dollar  and  case  error  rates.  The  dollar  error 
rates  are  projected  to  estimate  the  amount  of  dollars  underpaid  or 
overpaid. 

The  Office  of  Assessment  uses  the  statistics  generated  from  the  data 
base  to  prepare  reports  on  Rsi  payment  accuracy.  These  reports  are  to  be 
used  by  ssa  managers  to  provide  an  indicator  of  how  well  ssa  is  attaining 
overall  quality  objectives. 


Our  objectives  were  to  determine  (1)  whether  ssa  accurately  measures 
and  reports  RSi  payment  error  rates  and  (2)  the  total  amount  and  dura¬ 
tion  of  payment  errors.  We  focused  on  the  error  rates  ssa  reported  to  the 
House  Appropriations  Committee  since  that  was  the  most  current  infor¬ 
mation  provided  to  the  Congress  that  specified  rsi  error  rates. 

To  determine  if  payment  accuracy  rates  that  ssa  reported  represented 
the  errors  ssa  had  found,  we  obtained  ssa’s  payment  accuracy  data  base 
for  fiscal  years  1981-86,  generated  statistics  using  ssa's  criteria  for 
reportable  error,  and  compared  the  statistics  with  those  found  in  s.sa's 
reports  to  the  Committee  and  in  reports  within  ssa  (internal  Office  of 
As.se.ssment  reports  to  the  Commissioner  and  other  s.sa  managers). 

Where  w('  identified  instances  of  errors  that  s,sa  had  detected  and 
recorded  in  the  data  base  but  did  not  enter  into  the  calculation  of  its 
accuracy  rate,  we  recalculati'd  th('  payment  at'curacy  statistics. 

'I'o  d('termine  if  .s,sa  quality  re\  iew  analysts  might  be  mi.ssing  soiik' 
(*rrors.  thereby  understating  tlu*  incidenci'  of  ('rror.  we  analyzed  fiscal 
years  (  1  !>H1 -Ho )  of  payment  accuracy  rates.  I  sing  a  statistical  mod('i.  we 
comjiai'ed  all  possit)le  paii  s  of  re\  iewing  n'gions  to  see  if  errors  were 
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detected  by  each  region  at  the  same  rate.  This  comparison  was  possible 
because  each  region  reviews  some  ca.ses  from  every  otie  of  the  six  l•s(  s. 
Differences  in  rates  of  error  detection  were  tlien  tested  for  statistical 
significance  to  evaluate  whether  the  differences  were  in  fact  due  to 
chance  rather  than  to  some  other  factor,  sucti  as  ttie  adeptnes.s  of  ttie 
analysts. 

To  measure  how  errors  affect  beneficiaries,  we  reviewed  2.242  cases  (in 
.ss.-\’s  fi.scal  year  1984  payment  accuraev  samphM  in  detail  and  compiled 
cumulative  statistics  about  the  errors  in  each  ca.s(',  for  example:  total 
amount  of  payment  errors,  total  benefits  paid,  total  number  of  months 
paid  erroneously,  and  total  number  of  months  paid.  The  1.984  {payment 
sample  was  chosen  becati.se  it  was  the  latest  samitle  comj)l(>t('d  when  we 
began  our  review. 

In  this  report,  we  pre.sent  information  from  our  review  ;md  anal\  sis  of 
S.SA  records  (particularly  its  payment  accuracy  data  base),  interviews 
with  S.SA  officials,  statistical  projections  of  error  rates,  and  tt'chnical 
analysis  of  regional  differences  in  detecting  errors.  In  .May  1.987.  we 
briefed  the  staff  of  the  IIou.se  Appropriations  Committee  about  our 
review  of  ssa's  payment  accuracy.  Subsequently,  the  ('ommittee 
included  language  on  this  matter  in  its  1988  appropriations  report  (no. 
l()()-25(>).  The  Committee  advi.sed  .s.sa  to  di.sclose  information  on  its  inci¬ 
dence  of  errors  and  those  attributable  to  ssa,  as  well  as  the  perctmtage  of 
dollars  paid  erroneously. 


We  did  our  work  at  ssa  headquarters  in  Haltimore:  a  regional  asse.ssment 
office  and  satellite  office  in  Philadelphia;  and  the  Mid-.Mlantic  ;ind 
Great  Lakes  I’scs.  To  understand  the  three  routine  rsi  studies  (mentioned 
earlier)  u.sed  to  monitor  program  quality,  we  obtained  information  on 
tliC  quality  assurance  process.  We  also  gathered  and  evaluated  more' 
detailed  information  concerning  thest'  studies,  .\dditional  discussion  of 
our  scope  and  methodology  can  b('  found  in  appendix  1. 


We  did  not  review  computi'r  controls  relating  to  tlie  geia'ration  of  statis- 
tir's  from  the  data  basic  rtus  was  unm'ci'ssar\’  Ix'cause.  as  pai't  of  our 
d('t ailed  work.  w(>  nwii'wi'd  the  data  in  lt)84  case  fik's  ami  compari'd 
that  data,  on  a  case  by  ease  basis,  with  tlu-  information  in  the  data  base; 
using  s,s,\  estimating  I'rocedui'i's.  we  wi're  abh'  to  compute  tlx'  statistics 
reported  at  .s,sa  ( internally  )  and  outsidi' ,s,sa  (externally).  We  did  our 
work  from  March  l!)8.o  through  Octohi'r  198(1  in  accordance  with  gener 
ally  acec'pted  governmeiit  auditing  standards. 
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ssa's  measurement  system  for  the  lisi  program  is  adt'quate  in  that  it  ran 
provide  accurate  and  reliable  estimates  on  thi-  amount  of  error  in  the 
program.  This  measurement  system  uses  the  annual  Payment  .\(  i  iirac  \ 
Study  (mentioned  earlier)  to  draw  a  sample,  following  a  sjH'eified  sam¬ 
pling  methodology,  and  to  provide  basic  data.  These  data  are  projected 
to  the  universe  by  specified  statistical  techniques.  Although  the  mea¬ 
surement  system  is  technically  sound,  we  believe  the  data  could  be  bet¬ 
ter  presented  to  (1 )  more  fully  disclose  th?  amount  of  error  in  the  Rsi 
program  and  (2)  enable  managers  to  better  gauge  the  extent  to  which 
potential  improvements  might  be  realized. 

During  fiscal  year  1986,  ss.^  reported  to  the  Hou.se  Appropriations  ('om- 
mittee  an  rsi  accuracy  rate  based  only  on  dollars  paid.  e\en  though  ss.a  s 
Office  of  Assessment  had  also  measured  the  Ksi  program  with  a  rate 
based  on  cases  paid.  .ss.a  said  it  reported  an  accurac\  rate  ba.sed  onl\  on 
dollars  paid  because  that  was  what  it  had  reported  in  the  jiast  to  those 
inquiring  about  this  matter.  An  Rsi  rate  based  on  dollars  jiaid  generates 
a  high  accuracy  rate  and.  conversely,  a  low-error  statistic  because  the 
program  payment  errors  that  do  occur  in  monthly  payments  tend  to  be 
small  relative  to  the  total  benefit.  The  case  accuracy  rate  is,  therefore,  a 
better  indicator  of  the  incidence  of  error.  Both  rates  are  valuable  for 
describing  the  RSi  program  and  should  be  used  to  fully  disclose  the 
extensiveness  of  payment  errors. 

Dollar  errors  detected  by  ssa  analysts  are  about  twice  what  s,sa's  reports 
to  the  Committee  indicated,  ssa  counted  certain  overpaid  dollars  while 
excluding  several  categories  of  detected  errors  because  it  did  not  con¬ 
sider  these  as  reportable  errors.  In  addition,  s.sa  did  not  count  underpaid 
dollars  as  errors  even  though  they  were  detailed  in  internal  reports. 

For  fiscal  year  1986,  all  ksi  payment  errors  detected  by  ssa  analysts 
indicated  that  over  $1.1  billion  was  underpaid  or  overpaid  during  the 
year  and  about  4.2  million  persons  were  affected  by  payment  errors 
each  month,  rsi  payment  errors  may  be  even  greater  because  analysts  in 
some  regional  assessment  offices  may  be  better  at  detecting  payment 
errors  than  analysts  in  other  regional  assessment  offices. 


SSA  Does  Not  Report 
All  Detected  Errors 


SSA  reported  to  the  Committee  that  the  usi  accuracy  rate  ranged  between 
99.5  and  99.6  percent  from  fiscal  years  1981  through  1986.  This  figure 
includes  only  benefit  dollars  as,sociated  with  certain  overpayments 
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occurring  in  the  sample  month.'  It  does  not  include  all  detected  t'rrors 
recorded  in  the  data  base,  namely  net  underpayments,  errors  below  S.") 
per  month,  and  errors  barred  from  correction. 

Although  SSA  only  reported  certain  overpaid  dollars  to  the  Committee, 
internal  ssa  reports  specify  dollars  attributable  to  both  overpayments 
and  underpayments,  ssa  does  not  consider  underpayments  to  be  errone¬ 
ous  payments  because  underpayments  involve  instances  where  a  pay¬ 
ment  was  n^  made,  .ssa  said  it  did  not  report  underpayments  becau.se 
historically  it  had  reported  only  overpaymt'nts.  In  a  benefit  program 
such  as  Hsi,  we  believe  that  underpayments  should  be  reported  as  errors. 

An  Office  of  Assessment  official  stated  that  the  minimum  amount  for 
reporting  a  monthly  benefit  error,  set  at  $5,  was  based  on  the  quality 
assurance  criterion  used  in  the  Aid  to  Families  With  Dependent  Children 
program.  This  $5  amount  represents  a  negotiated  criterion  agreed  to  by 
the  federal  government  and  the  states  for  determining  state  fiscal  liabil¬ 
ity  for  incorrectly  paid  benefits,  ssa  decided  to  adopt  this  criterion  for 
the  Payment  Accuracy  Study,  even  though  determining  fiscal  liability  is 
irrelevant  to  this  study. 

A  further  justification  given  by  s.sa  for  using  a  $.5  exclusion  criterion 
was  that  inclusion  of  errors  of  less  than  .$o  per  month  would  not  signifi¬ 
cantly  affect  the  error  rate.  We  found  ssa's  position  to  be  correct  about 
the  dollar  error  rate  (.see  p.  20);  however,  inclusion  of  errors  ranging 
from  $  1  to  .$5  a  month  affected  the  case  error  rate  by  an  average  30 
percent  (.see  p.  17);  therefore,  in  our  opinion,  these  U'sser  amounts 
should  be  included.  Further.  .s.sa  recognizes  that  t)H'S('  are  paym(>nt 
errors  even  though  not  reported;  they  must,  therefore,  be  corrected 
unle.ss  they  are  barred  f?-om  correction.  We  did  n(»t  consider  beiu'fit  c<»m- 
putation  errors  below  .$1.00  to  be  payment  ('rrors  tu'cause  s.sa's  rounding 
procedures  for  benefit  payments  require  rounding  computations  down¬ 
ward,  not  upward,  to  the  nearest  dolhir. 
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The  Payment  Accuracy  Study  procedures  state  that  if  a  payment  case  is 
barred  from  correction  by  an  administrative  proceduri',  it  is  to  be 
repotted  internally  as  a  “non-dollar  impact  deficiency."  .ss.\  believes  that 
reporting  cases  barred  from  correction  as  errors  is  inappropriate 
because  the  cases  are  deemed  “correct”  by  an  administrative  procedure. 
SSA  also  believes  that  to  report  such  cases  as  payment  errors  would  con¬ 
tradict  this  procedure.  We  did  not  evahiate  the  merits  of  the  administra¬ 
tive  procedure,  which  also  affects  s.sa  programs  that  we  did  not  review. 
However,  in  our  opinion,  cases  barred  from  correction  should  bo 
reported  as  payment  errors  because  the  payments  for  these  cases  are 
different  from  the  payments  that  would  have  been  made  had  the  cases 
been  handled  correctly  from  the  beginning. 

s,sa's  including  all  errors  in  calculating  and  reporting  payment  accuracy 
would  better  indicate  the  usi  program's  overall  performance;  it  would 
enable  ssa  to  better  gauge  the  extent  to  which  potential  improvements 
might  be  realized.  An  overstated  high  accuracy  rate  could  lead  managers 
to  conclude  that  further  improvements  would  be  marginally  cost-effec¬ 
tive  and  therefore  should  not  be  explored.  Conversely,  management  may 
have  more  incentive  to  improve  an  accuracy  rate  of  85  percent,  for 
example,  than  one  of  92  percent.  Fully  disclosing  the  extent  of  all  errors 
occurring  would  enable  managers  to  better  decide  whether  the  causes 
should  be  more  closely  examined  so  as  to  reduce  future  incidence.  Like¬ 
wise,  fully  disclosing  all  errors  to  the  Congress  should  enable  it  to  better 
carry  out  its  oversight  responsibilities. 


Reporting  Error 
Incidence  and  Causes 
Would  Better  Indicate 
SSA  Performance 


s,SA  reported  dollar  error  rates  to  the  Committee  and  in  internal  Office  of 
Assessment  reports.  However,  the  internal  reports  also  included  the 
related  case  error  rates  and  data  on  the  extent  of  error  attributed  to  ssa 
and  others:  such  rates  and  data,  which  were  not  reported  to  the  Commit¬ 
tee,  provide  a  measure  of  the  extent  to  which  beneficiaries  are  affected 
by  errors  and  the  extent  to  which  ss.\  is  responsible.  These  reports  are 
not  routinely  distributed  outside  ssa.  According  to  an  ssa  official, 
neither  the  Secretary  of  Health  and  Human  Services  nor  the  Congress  is 
sent  the  reports  because  ssa  believes  neither  desii-es  such  detail.  We 
were  told  that  internal  reports  have  been  distributed  outside*  .s.sa  when 
requested  or  when  s.sa  felt  the  reports  answei'cd  a  specific  inquiry. 


Case  accuracy  rates  are  a  better  indicator  of  s.s.\'s  ability  to  pay  benefits 
accurately  bt'cause  these  rates  show  how  many  people  are  being  cor¬ 
rectly  or  incorrectly  paid,  whereas  the  dollar  acciu'acy  rates  are  a  mea¬ 
sure  of  the  size  of  dollar  errors  in  relation  to  total  dollars  paid,  .Mthough 
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the  latter  are  an  appropriate  measure,  they  mask  the  frequency  of  error. 
For  example,  if  half  of  all  beneficiaries  were  underpaid  $  10  per  month 
for  1  year,  the  dollar  accuracy  rate  would  be  correctly  calculated  as  99 
percent  (based  on  an  average  payment  of  $500)  and  the  case  accuracy 
rate  as  50  percent.  The  dollar  accuracy  rates  show  that  the  trust  funds 
are  not  significantly  affected;  the  case  accuracy  rates,  that  many  benefi¬ 
ciaries  are.  Both  rates  are  valuable  for  program  oversight. 

Case  error  rates  for  fiscal  years  1981-86  doubled  from  an  average  of  7.9 
percent  (as  reported  in  internal  .ssa  reports)  to  14.9  percent,  if  all  errors 
identified  by  .ssa  analysts  and  recorded  in  the  data  base  are  counted. 
Similarly,  dollar  error  rates  doubled  from  an  average  of  about  1/2  of  1 
percent,  as  reported  to  the  Committee,  to  almost  1  percent  (if  underpay¬ 
ments,  as  well  as  certain  other  overpayments  not  now  reported,  are  con¬ 
sidered  erroneous).  Including  all  errors  identified  when  calculating 
accuracy  rates  would  not  incur  additional  cost  because  all  errors 
detected,  regardless  of  whether  they  are  reported  as  errors,  are 
recorded  in  the  data  base  to  provide  a  record  of  what  was  found. 

ssa’s  Payment  Accuracy  Study  is  not  intended  to  provide  data  on  the 
causes  of  case  errors,  and  the  size  of  the  annual  sample  does  not  permit 
projecting  specific  causes  with  statistical  certainty:  however,  for  inter¬ 
nal  reporting  purposes,  ssa  does  categorize  the  causes  it  identifies.  This 
categorization  distinguishes  between  errors  caused  by  s,sa  and  others. 
Generally,  .s.sa  causes  most  underpayments,  ranging  from  a  Icav  of  72. 1 
percent  in  fiscal  year  1986  to  a  high  of  96.1  percent  in  fiscal  year  1982: 
others,  such  as  beneficiaries,  cause  most  overpayments,  ranging  from  a 
low  of  50.0  percent  in  fiscal  year  1986  to  a  high  of  74.7  percent  in  1983. 
Reporting  and  comparing  the  incidence  of  error  caused  by  s,sa  each  year 
would  more  precisely  indicate  how  well  ssa  fulfills  its  responsibility  to 
accurately  pay  benefits. 

Taking  the  errors  detected  by  ssa  in  the  annual  sampU\  we  calculated 
who  was  responsible  for  payment  erroi's.  Overall,  most  errors  were 
attributed  to  ssa;  the  ca.se  error  rate  decreased  between  fiscal  \'ears 
1982-85.  showing  a  slight  but  statistically  insignifit  ant  increase  in  li)S6 
(see  table  2. 1 ). 
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Table  2.1:  Causes  of  Payment  Errors 

(Fiscal  Years  1981-86)* 
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In  percent 


Fiscal  years 


Cause  of  error 

1981 

1982 

1983 

1984 

1985 

1886 

SSA 

11  5 

14  4 

100 

8  1 

62 

70 

Beneficiary  or  other 

20 

2  1 

29 

37 

26 

30 

■’Excludes  errors  made  in  cases  barred  from  correction  by  SSA  s  admimstralivo  procedures 

Some  errors  could  be  categorized  as  being  caused  by  either  t  he  benefici¬ 
ary  or  ssA,  depending  on  how  one  interprets  the  findings  in  a  specific 
case.  For  example,  a  beneficiary  who  was  overpaid  because  he  did  not 
report  earnings  in  excess  of  the  maximum  allowed  alleged  he  had  not 
been  told  that  such  earnings  had  to  be  reported,  ssa  says  it  advises  all 
beneficiaries  of  this  requirement  at  the  time  of  application  and  sends 
periodic  subsequent  reminders;  in  addition,  this  requirement  is  stated  in 
the  information  provided  to  applicants  and  beneficiaries,  ssa  attributed 
the  error  in  the  example  abovt'  to  the  beneficiary.  In  reviewing  ssa's 
basis  for  categorizing  the  causes  of  errors,  we  found  that  its  determina¬ 
tions,  generally,  were  reasonable  and  based  on  the  information  in  the 
file. 

Portraying  the  incidence  of  error,  as  shown  in  table  2,1.  gives  a  clearer 
indication  of  s.sa’s  performance  over  time;  however,  because  of  the 
nature  of  the  annual  Payment  Accuracy  Study,  incidence  of  error  is  not 
the  best  indicator  for  detecting  recent  changes  in  accuracy.  The  study 
measures  cases  being  paid  erroneously  as  of  the  month  of  the  study; 
therefore,  it  primarily  reflects  errors  made  in  earlier  years.  Conse¬ 
quently,  the  usefulness  of  the  data  in  measuring  how  aicurately  ssa 
handled  actions  affecting  benefits  in  IPSti  versus  1985.  for  example,  is 
limited.  Better  measures  for  that  purjio.se  are  studies  of  process  accu¬ 
racy — the  Awards  and  Disallowanci's  Study  and  the  Postadjudicative 
Study.  Such  measures,  which  are  compiled  as  a  (i-month  rolling  average 
and  reported  quarterly,  would  be  better  indicators  of  the  effect  staff 
reductions  could  be  having  on  Ksi  jirogram  accuracy.  Both  the  annual 
payment  and  quarterly  process  accuracy  rates  are  included  in  our  three 
reports  on  Social  Security.  ’ 
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Chapter  2 

RSI  Payment  Errors  Higher  Than  Reported 

RSI  Cases  in  Error 

ssa’s  detected  and  reported  case  error  rates  over  6  years  are  shown  in 
figure  2.1.  Cast'  error  rates,  reported  internally,  would  increase  if  cases 
with  a  monthly  dollar  error  below  $5  and  cases  with  errors  barred  from 
correction  by  an  administrative  procedure  were  included.  For  instance, 
in  fiscal  year  1986,  the  case  error  rate  increased  from  a  reported  5.9  to 
12.7  percent  when  all  detected  and  recorded  errors  were  counted.  Each 
case  represents  one  or  more  beneficiaries  paid  under  one  Social  Security 
account  number.  The  categories  represented  by  the  bar  graphs  in  figure 
2.1  include  both  overpaid  and  underpaid  cases  because  ssa  includes  both 
in  its  internally  reported  statistics. 

1981-86) 

21  Percent  of  Cases 

1981  1982  1983  1984  1985  1986 

Fiscal  Year 


I  Adminisiraiiveiy  Barred  Triors 

^  J  Monrhty  L  rrors  Below  $5 
SSA  I  "Oi  Cidt'f  a 


P’rom  fiscal  years  1}»81  throuj’h  U)8(i,  ssa  inti'rnally  reported  case  error 
rates  that  averajled  7,8  percent.  When  errors  helow  $o  per  month  and 
errors  barred  front  corri'ction  are  addt'd,  thi'  avera^Ji'  I'rror  r;ite 
incH'ases  to  14.8  percent.  The  error  r;tt('  di'crt'itsi'd  from  1882  to  l8S.'i 
larj'ely  because  of  fewer  cast's  beinji  underpiiid. 
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Chapter  2 

RSI  Payment  Errors  Hi({herThan  Reported 


We  tested  the  changes  in  case  error  rates  for  statistical  signif  icanct'  to 
determine  whether  the  changes  could  be  due  tt)  chance,  inherent  in  ran¬ 
dom  sampling.  We  found  that  the  increase  from  1981  to  1982  and  tlu' 
decrease  since  1982  were  not  due  to  chance,  but  we  cttuld  not  rult'  out 
that  possibility  for  the  small  increa.se  frttm  1985  to  198(), 

The  underpaid  ca.se  error  rate  dropped  from  14.8  perci'iit  in  1982  to  (>.7 
percent  in  1985.  .ssA  attributes  the  underpayment  decri'a.s('  to  more 
timely  processing  of  benefit  increases  from  earnings  after  retirement.' 
The  error  rate  increa.sed  slightly  in  198()  because  the  number  of 
underpaid  cases  below  $5  per  month  went  up.  ss.\  was  responsibU'  for 
72.1  percent  of  the  underpaid  cases  in  1985,  primarily  because  of  lom- 
puter  processing  problems.  The  percentage  of  underpayments  causi'd  by 
SSA  has  generally  declined  from  a  high  of  95. 1  percent  in  1982. 

From  1981  through  198().  overpaid  cases  fluctuated  between  4.7  and  5.() 
percent  of  cases  paid.  liefore  1986,  beneficiaries  had  bi'en  responsible 
for  the  greater  proportion  of  overjjaid  cases.  This  proportion  peaked  in 
1983,  when  beneficiaries  were  responsibk'  for  74.7  percent  of  the  over¬ 
paid  cases,  but  declined  to  50.0  percent  in  1986.  Of  the  cases  it  analyzed 
and  reported,  ssa  found  that  the  reason  for  this  dc'cline  was  ( 1 )  that  ben¬ 
eficiaries  were  more  accurately  reporting  earnings  to  comply  with  the 
annual  earnings  test  and  (2)  the  phasing  out  of  student  benefits.  s.sa 
attributed  the  improved  accuracy  in  earnings  reporting  to  its  sending 
reminders  to  beneficiaries  about  estimating  earnings. 


■*In  ordor  to  kn<m  if  KSI  tx*nofUs  should  Ix'  imrraM^d  tor  t  hns«*  cut  it  lot  1  in  i  u  Im  oarn  \s  a^os  nr 
have  M*if-cmployt‘<i  incomo.  thr  Sn-ial  S«*<  uniy  Act  rrtjuin's  SSA  in  n-t  nniimin  i  heir  hcnrfit.s  SSA  s 
pxilicy  i.s  to  Htintially  rrviovv  lM*tu*fn  r<»mput  alums  lor  iM'nofuianos  wit  h  rot  nil  oarninvl''  A  |»anially 
aiJtomatf’d  pna-oss.  Automatic  KarmnUs  Koappraisal  t  JjM'ratioii  is  iisod  it*  rot  «*mpulo  U-not'ii 
amounts  Thotimoii  imk  to  comploto  this  pnM-oss  mcroasod  from  lomtuiihsm  lUTS  in  42  nit>nths  m 
1980;  aft<*r  1980  tho  iim«*  roquin*d  has  jiraduaDy  lx*on  roducod  Tfio  SSA  tinio  t  niorittii  fnr  a  paynu*nt 
.samplo  error  is  that  the  pm  oss  should  lx‘  comploiotl  nt»  laior  i  ban  2 1  moni  hs  after  1  lit'  end  t*f  Uio 
earnings  year  (tA(  Vs  IKdays  in  Kecomputing  So<  lal  J^  untv  lk*nol ils  (  aiist'  1  Mdoq*a\  rnonis  ft>r 
Kxtended  IVruKls i(i  ,\0  TfRh  ftm  -  S'pt  10.  1984  '  disoussos  prohloms  SS.\  oxjH’rioni  od  in  t  bis  anxi 
dtinrig  1978-82 

SK’ial  S'ciinty  Nuiofils  are  inoaiil  it*  I'oplaco  m  pan  oanungs  Insi  in  a  Itonoli*  iar\  nr  t'onoficiarios 
IwcauM*  <tf  rotiromrnt  disability  <»r  <  loath  Thorob»ro  t  )io  anmunt  of  Unofil^  l  bat  a  rol  inn-  uni  lor  ago 
7o  and  do|M*ndonis  will  rot  oi\  oa<  h  year  doj>ontis  nii  u  hot  bor  i  lio  l*onofii  iar\  ix  pan  iall\  nr  full\ 
rotii'i’d  The  oamings  lost  i  alst»  knttwn  its  iho  roiironioni  lost  !  is  used  to  nio.isiiro  ibo  dogroo  n|  roliri' 
mont  and  ft»  tfoformino  tno  umoiinf.  if  ari>  ,  t<f  U'  dot  lot  tod  (f  uni  m<  mi  bl>  l>onohis  Annual  o\onif*! 
amountsvarN  doi>oM<ling  t»n  \\  holhor  the  U*no!n  iar\  is  mulor  ago  ot  at  loa.si  ago‘>’  Amnunisalsn 
\  ar\  from  \oar  to  year  to  keep  lh«‘m  in  ratio  witb  oarniiigs  lov*-ls  Ik  notu  lai  tox  are  roumrod  tn  osu 
mate  earnings  tor  the  *‘oming  year  and.  d  ntsossarx  fuMiotVs  are  .nliiixfni  based  mi  rfio  osi iniato 
Ft)llo\^ mg  the  end  t *f  tbo  earnings  voar  honofi<  larios  an-  roijiiiroti  t n  I  ilo  an  annual  i  I'l**  iri  sj>im  il  \  mg 
the  exact  amount  earned  during  ibo  prov  huix  \ oar  <intl  i!  nooilod  a  r * au  iltai  inn  < 4  hi  iiot iis  i o*  oi\  o*l 
is  m.ido 
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RSI  Payment  Krrors  Higher  Than  ReiKtrted 


I'sing  th('  cast'  orror  ratt's  just  (iiscusst'd.  wu  (‘stimated  the  numbiT  of 
casos  paid  iu  error  oacli  month  in  a  tfiveii  year,  from  lfi81  to  The 
ostimatt'd  number  of  cases  paid  incorrectly  o\'('r  b  years  is  sliovvn  in  fig¬ 
ure  2.2. 


Figure  2.2;  Cases  in  Error  (Fiscal  Years 
1981  86) 

6  Millions  of  Cases 


1981  1982  1903  1984  1905  1906 

Ftscal  Year 


A'l"  •  M'  v*-y  -••0  (  "O's 

i  M-''  ■  V  I  "O-n  $.•" 

lill  S'.A  I 

I  siiic  ss\'s  (Titerion  for  re))or1  in^  an  erroi .  an  a\ cra^t'  of  2  million  cast's 
was  eifoui'ously  paid  each  year  o\  t'r  t  ht'  (>  year  jteriod.  I  lowevt-r.  when 
oI  her  cale^orit's  of  recttrded  errors  are  iiu  ludt'd.  t  he  a\  t'l  agt'  rises  to 
almosl  1  million  cases  per  >  ('ai’  1  sin^  1  he  a\ cra^te  numirer  of  beiu'fi- 
ciaiaes  per  i;sl  case,  we  estimate  that  the  a\  t'ra^t'  numbt'r  of  persons 
affected  b>  [tayinenl  errors  was  about  1.7  million  pei' year  foi' t he  li-,\ t'ar 
pentrd  For  Ib.Sb,  anesiimaleti  1.2  million  pt-rsons  wt'it' affect t'd  b,\  pa,\ - 
meni  errors 
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RSI  Benefit  Dollars  Paid  in 
Error 


Table  2.2:  Dollar  Error  Rates  Over  6 
Fiscal  Years' 


The  dollar  I'rror  rate  ealeulated  by  ssa,  used  to  report  payint'nt  aeeuraey 
to  tlu'  t'ongress,  averaf»ed  less  than  1/2  of  1  pi-rcent  over  (>  fiscal  years. 
We  compared  this  rate  with  the  dollar  error  rat('  we  calculat('d  that 
includ('d  additional  ssa  drUected  eri'ors  (tabU'  2,2).  Consideration  of 
th('S('  additional  error  catejtories  inci'easi's  the  averafte  dollar  (>rror  rate 
to  almost  1  {)ercent.  A  dollar  error  rate  of  1  percent  seems  low,  but  cur¬ 
rently  this  rate  amounts  to  $1  .o  billion  in  erroiu'ous  benefits  each  year. 
Of  the  three  types  of  errors  ksa  detected  but  did  not  include  in  its  dollar 
('iTor  rate  calculation,  including  underpayments  as  a  dollar  error 
increased  the  rate  by  the  largest  increment.  Overall,  from  1981  to  1983, 
the  dollar  error  rate  steadily  im  rea.sed  b('cau.s(>  of  increa.sing  underpay¬ 
ments;  the  rate  then  decreased,  although  not  constantly,  with  the  lowest 
total  dollar  error  rate  for  the  b  years  in  1981). 


In  percent 


Recorded  errors  1981 

Calculated  by  SSA  0  52 

Additional  calculations  by  GAO 

Underpayments  0  37 

Below  $5  per  month  0  03 

Barred  0  13 

Total  1,05 


Fiscal  yea^s 


1982 

1983 

1984 

1985 

1986 

0  43 

0  52 

0  49 

0  48 

0  35 

0  58 

0  59 

0  27 

0  36 

0  19 

0  02 

0  01 

0  02 

0  02 

0  02 

0  14 

0  09 

0  04 

0  07 

0  09 

1.17 

1.23 

0.83 

0.96 

0.66 

’We  calcolatod  CffOf  rates  usin<}  SSA  cl  df'tot.lod  arid  rccoffled  f’fiors  I  he  total  error 

rates  are  weighted  estins-ilos  oot  the  ,jfifhnie?i<  so'n  ot  ttu*  trulivulual  i  oropi.tni'tits 


The  efh'ct  of  differing  error-reiiort  ing  criteria  on  project I'd  error  dollars 
is  illustrated  in  figure  2.3.  For  examjrle.  in  fi.scal  year  198(1,  the  ('sti- 
mated  dollars  paid  in  error  grew  from  $1)02  million  ($().()  billion),  using 
the  accuracy  rate  ssa  reported  to  the  Committee,  to  $1.1  billion  when  all 
detected  and  recorded  errors  are  included.  In  this  figure,  underiiayments 
are  illustrated  separately  because  ssa  did  not  include  them  in  its  report 
to  the  Committee,  ssa's  198(1  internal  report  shows  underpaid  dollars 
and  overpaid  dollars  in  a  similar  manner.  .\s  shown  in  figure  2.3,  the 
addition  of  errors  of  less  than  $3  ixu'  month  doi's  not  materially  affect 
tlu'  projected  dollars  in  error,  even  though  the  addition  of  thesi'  errors 
does  significantly  increase  the  pro.ji'cted  cases  in  error  as  was  seen  in 
figure  2.1, 
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Figure  2.3:  Dollars  in  Error  (Fiscal  Years 
1981-86) 


2.0  Billions  of  Dollars 


1981  1982  1983  1984  1985  1986 

Fiscal  Year 


i  j  Ad-’i.c.sif.irvo  r  O.T'oc  (  "O's 

!_  ”]  Mo-r'y  t  "O'S  Hf  ow  $5 

Ir'V'na^y  lli'oo'U’C  U''0r'yay"''0'':s 
SSA  Hcyo'V'O  Ov^'o-iy '"(>*' 's 


••Xccording  lo  ss,\,  it  was  n'sponsibU'  tor  77  1  pcrcont  of  the  undorpaid 
dollars  in  IftHC).  IncornM  l  coiiiputations  hav('  ri'placod  late  nromiiula- 
tionstas  monlioiK'd  oarlior.  to  incroaso  biMUd'its  as  a  result  of  earnings 
after  retirement )  as  tlie  leading  under})ayment  probU'm. 

In  198().  SSA  r<>i)orted  that  lH>n(>ficiaries  were  responsibk*  for  o.'i.7  per¬ 
cent  of  the  overpaid  dollars.  Although  the  annual  earnings  test  was  still 
the  highest  singU'  rea.son  for  overpayments  (  L'l  .0  percent  ).  this  reason 
was  at  its  lowest  point  since  overirayments  have  been  rej'oi'tcd  Most 
overjiaid  dollars  for  ItbStl  resulted  from  probU-ms  that  fornu'ti.N  had  a 
much  smaller  dollar  error  rate,  such  as  incori'ect  determinal  ions  of  fam¬ 
ily  relationships. 
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Payment  Accuracy 
Reviews  May  Not  Be 
Detecting  All  Errors 


Table  2.3:  SSA  Regional  Assessment 
Office  Error  Detection  Rates  by  Sample 
Year 


Chapter  2 

RSI  Pa>  meiit  Errors  Higher  Than  Reian-ted 


Quality  review  analysts  determine  payment  accuracy  rates  by  rt'vit'win^ 
cases.  We  found  tltat  analysts  in  some  rt'gionai  assessment  offices  may 
be  better  at  detectinj*  errors  tiian  analysts  in  otlu'r  regional  assessment 
offices.  Consequently,  Rst  errors  may  be  higher  than  indicated  in  this 
report.  We  combined  years  of  payment  accuracy  rt'stilts  to  compare 
the  10  regioital  asst'ssment  offices.  Our  statistical  analysis  showed  sig¬ 
nificant  differences,  which  could  not  be  attributed  to  chancc',  in  tlu' 
numbers  of  errors  detected  by  the  various  regions.  'I'hese  differences 
were  not  detected  by  the  headquarters  group,  which  sampk's  and  e\  alu- 
ates  ca.ses  reviewed  by  the  regional  assessmimt  offices.  We  apiilied  an 
analysis  methodology  specifically  dc'signed  to  identity  statist icall\’  sig¬ 
nificant  differences  in  detection  rates  between  i)airs  of  regional  offices. 
s,s.A  did  not  make  such  an  analysis. 

Each  regional  office's  error  detection  rates  for  thi'  paymc'nt  accuracy 
samples  of  1981-85  an*  shown  intabl('2.3.  For  ('xtimplt'.  in  1981.  the 
Ho.ston  Regional  Office  found  17,2  percemt  of  the  ca.si's  it  ri'viewed  to  b(' 
in  error.  In  the  1981-85  period,  overall  dc'ti'ction  rate's  among  tlu'  offices 
ranged  from  a  low  of  1 1 .8  percent  for  Denvt'r  to  ;i  higli  of  Itl.ti  percent 
for  Philadelphia.  Some  of  the  diften'nces  in  I'rror  de'tection  rates  are 
because  the  majority  of  sample  c.ist's  assigned  to  it  regiontil  office'  fen- 
review  ceime  freim  erne  eir  t  we>  e)f  the'  six  ese  s;  t  hits.  I  he'se'  e  ase's  mii\  neet 
have  the  same  frequency  of  erre>rs  as  theese'  e  ase's  assigne'ei  tee  aneethe'r 
re'giemal  eiffice,  whie-h  re'ceives  e-ase's  freem  diffe're'iit  ese  s.  State-el  aneethe'r 
way,  a  re'gional  office'  assignc'd  ca.sc's  with  more'  e'crors  should  find  more' 
e'lreers. 


In  perce.'nl 


Regional  assessment  office 

1981 

Sample  year 
1982  1983 

1984 

1985 

5-year 

average 

B(.)yton 

1  7  v- 

Cl 

'  ■ 

\cv'.  Yiirh 

;  -1  e, 

an) 

Phil.Ktf.'l|)hi,.i 

P*  1 

tj 

14  ■ 

■  : 

All, ml, i 

K.  ’ 

■ 

K'  .  ' 

’  •>  y. 

1  i.ill.)'. 

K.in'„r,  i  :< . 

. 

"■  • 

'l  -t 

1,1!  I  M'ti  , 

1  1  ’ 

’■1  ■■ 

:  ■ 

M  f 

'  • , 

Overall  average 

16.6 

19,5 

15.3 

13  9 

113 

16  1 

I'anr  >2 
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Table  2.4:  Comparison  of  Regional 
Assessment  Offices’  Likelihood  of 
Detecting  Payment  Error 


The  study  results  shown  in  table  2.3  do  not  confirm  that  one  region  is 
consistently  better  at  detecting  errors  than  anotlu'r.  Differences  ( inild  lie 
attributed  to,  as  mentioned  above.  ( I )  some  regions'  being  assigned  more 
cases  with  errors  than  tho.se  assigned  to  other  regions  or  ( 2 )  the  normal 
and  expected  variation  that  arises  from  random  sampling  Siik  e  we 
wanted  to  know  whether  some  regions  were  better  at  di'teeting  errors 
than  others,  we  analyzed  the  1981-8")  Payment  .Accuracy  .'^tndy  data  to 
determine  if  differences  in  error  deti'ction  rates  bet\,een  n'gions  were 
statistically  significant."  Were  we  to  find  any  statistically  significant  dif¬ 
ferences,  we  would  conclude  that  some  factor,  other  than  the  sampling 
process,  was  affecting  error  detc'ction.  In  our  opinion,  that  factor  would 
most  likely  be  the  ability  of  the  regions  to  detect  errors  Our  statistical 
model  controlled  for  other  influences  by  mathematicall>  remo\ing  the 
effect  of  some  offices’  receiving  for  review  more  cases  with  errors  than 
others;  we  then  evaluated  the  statistical  significam  e  of  the  remaining 
differences  in  error  detection  rates.  ( 'sing  the  model.  w('  compared  tb.e 
differences  in  rates  between  all  possible  pairs  of  regions  and  found  com¬ 
parisons  with  Chicago  and  Denver  to  b('  statistically  significant.  Wi- 
found  no  other  comparisons  to  be  statistically  signific  ant.  From  tiu' 
model,  first,  we  conclude  that  Philadelphia  Kansas  City.  Dallas.  New 
York,  and  Boston  were  more  likely  to  detect  an  error  th<m  Chicago  (table 
2.4).  For  instance,  Philadelphia  would  have  an  85.3  percent  better 
chance  of  detecting  an  error  than  Chicago;  we  are  90  perccmt  confidcmt 
that  the  true  difference  ranges  between  82.3  percent  and  88.4  percent. 


In  percent 


Greater  probability  of 
detecting  payment  error 
than  Chicago 

Regional  assessment  office 

Estimate 

Interval 

Philadelphia 

85  3 

82  3  to  88  4 

Kansas  Cdy 

84  8 

76  7  to  9 1  6 

Dallas 

73  8 

70  7  to  76  9 

New  York 

68  7 

59  3  to  ^8  " 

Boston 

51  8 

20  6  to  9i  0 

^Intervals  a'e  comcmtcd  a!  me  90  Dc'ceei  p.o'  o'  sMSi'  oa  oc  '  oenoo 


'  DiflVn-nrrs  wiMT  cimsulc-ica  sinniliiain  it  ilii-v  run  ihrlio  pi.i.riit  IomI.m  vi  i,  ,,l  , ,  .i.t  lOi  m,  , 

ttinl  IS.  if  'vr  (Iri-w  lOd  sainplcs.  wo  wimid  I'Xpoi  I  In  timl  n  roi  dii  o.  i  ion  i  .iicdil  lonii.  r-  t  .lifi'e 
within  till’  nmfidomr  inicrval  in  iid  of  ihosc  x.iniiilo^ 
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c  hapter  2 

RSI  Fayineiit  KrTors  Higher  Than  Reported 


Sc'fonti,  \v('  tounci  statistically  significant  differences  in  I'rror  detection 
Ix'tween  the  Philadelphia  and  DenvtT  offices.  We  estimated  that  Phila- 
deljihia  had  a  72.0  percc'nt  better  chance  of  delectinf'  an  error  than  Den- 
\CT  if  it  reviewed  the  sante  case,  and  we  are  00  percent  confident  that 
thi'  trill'  differt'ncc'  lit's  between  4H.8  and  08.7  percent. 

Although  our  analysis  su^gt'sts  that  somt'  rc'vit'wers  are  not  deti'ctin>» 
some  errors.  ss.\  lu'adquarters’  re-rc'views  of  a  sample  of  cases  reviewi'd 
by  ri'j'ional  assc'ssmc'iit  office's  genc'rally  have  not  found  that  reviewers 
missed  I'rrors.  s.s,\  hc'adquarters  us-'s  a  subsample  of  the  KSI  sample  ca.st's 
to  validate  the  rc'sults  of  list  paynient  accuracy  case  revit'ws  done'  in 
re^*ional  office's.  After  a  case  is  se'h'cte'ri  for  re-revit'w  by  headquarters 
personnel,  the  claims  foleler  and  the  quality  a.ssurance  review  folder 
(with  evieience  gatlu're'd  by  ri'f’ional  quality  review  analysts)  are  sent  to 
ss.\  headquarters  where'  pe'rsonni'l  review  the  regional  findings  and  the 
coding  of  the'se  findings  in  the  data  base.  Headquartc'rs  personnel  have 
only  paperwork  available  for  tht'ir  review,  wheri'as  the  regional  quality 
review  analysts  had  the  opportunity  to  intervit'w  each  beneficiary  and 
others,  such  as  e'tnployers.  and  to  gather  additional  evidemce.  If  relevant 
evieience  was  not  gathered  during  the  regional  review,  the  subsample 
reviewer  is  limite'd  in  the  conclusions  to  be  drawn. 

F(  the  period  we  examined  ( 1981-85),  subsample  reports  show  that 
most  error  case's  and  a  sample  of  nonerror  cases  were  re-  reviewed  at  s.sa 
headquarters.  The  number  of  nonerror  cases  re-  reviewed  was  reduct'd 
during  this  period  bt'cause  ssa  believed  that  ( 1 )  review  accuracy  had 
increased  and  (2)  the'  amount  of  othe'r  work  done  by  the  ht'adquartt'rs 
re'view  group  had  increaseel.  Recent  cases  re-re'viewed  have  included 
most  e'l  ror  ease's  and  about  10  perce'nt  of  nonerror  cases;  but.  generally, 
the'  re'-review  has  not  ieli'iitifie'd  any  c'rrors  missed  by  regional  reviewers. 

SSA  could  consider  wlu'thc'r  it  would  be  cost-e'ffe'ctive'  to  have*  ca.ses  from 
each  office'  re'-re'vie'we'd  by  an  expe'rienci'd  re'viewer  in  anotlu'r  region.  If 
such  cross-clu'cks  were'  done  without  the'  be'iiefit  of  the  re-rev it'wer's 
knowing  the'  prior  de'termination.  the  cross-che'ck  could  provide'  an  inde- 
jie'nde'tit,  te'e-hnically  semnd  ve'rificat  ion.  Current  re'-re'vie'ws  by  head- 
quart  e'rs  pi'isonne'l  ai'e'  made'  with  knoe\  le>dge'  etf  the-  re'vie'wer  s 
de'te'rmination;  tlu'se-  re'-re'vie'Ws  te'iiel  to  e'tniihasize'  e-nsuring  that  the' 
re'viewe'f's  ceiding  e»f  finelings  is  e onsiste'iit  wit  h  iiroci'dures  rather  t  ban 
cross-che'cking. 
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(  hapter  2 

RSI  Pa>ni«‘»U  Krrors  lli^lirr 'Diaii  Krporirf) 


ssa's  paynu'iit  accuracy  tor  ksi  Itcnctus  is  iioi  as  ^(kkI  as  ihum  ra\  cii  ti\ 
rcjiorls  to  t i\c  Hous('  Apj)ropi'iatioiis  ( 'onmiii (cc  l''( ir  (tic  i>i  pro-  am.  "  ' 
rcport(‘(i  a  payment  accuracy  rate  no  li-s  than  It!'.-'  pi'rceni  loi-  ll'S]  sc. 

( )ur  analysis  of  itsi  payment  aciairai  \  iiuiu  ic".  ci-.  i  hai  r  lio 

reported  accuracx'  rates  o\-erstate  jieii'oi-iuaui  i’  tu'caii^i'  s^  \  dors  m  a 

count  all  of  the  errors  it  di'K'cls  in  >'ear)\'  studies,  c.m iudiii;t  ci  i  nr^ 
ui\  oK  me  I  1  )  underpayments.  ( 2 )  amounts  below  Sd  pi  a-  nioiii  h.  and  ■ 
casi'^  precluded  from  correction  because  of  admiiiist rat i\  e  proi  odiu 
and 

report  acciiracx'  rates  on  t  he  basis  of  cases  paid,  w  hich  i  includin.d  I  hr 
afio\  1'  three  cate>fori(‘s t  |■('sulIs  in  an  a\'eraf.;e  of  Hd-fiercenl  paynieni 
accuracy  for  IttKl-Sd. 

Not  iiu  ludinj'  all  errors  detecti'd  u  hen  c;ik  iilatin^  accuracy  rates  o\  er- 
stati's  program  performance  and  can  mislead  mana, tiers  and  the  Con¬ 
gress  about  th(>  e\t(‘nt  to  which  iiotential  impro\  emenis  mi^bt  be 
r('aliz(‘d.  In  addition.  ss.\  needs  to  determine  what  additional  monitoriii,U 
or  ^'erificatioti  mijilit  he  tteeded  to  ensure  (jualit  y  reciew  analysis  are 
detectiufi  till  errors. 

fundiuneutat  )ioal  of  ss.\  is  to  makt'  accurate  i;si  benefit  iia\'m('nls,  pro- 
ti'ctinji  t h(‘  iitten'sts  of  beneficiaries  and  1 1 ,e  intejirit\  of  the  trust  funds. 
.As  such.  tlu>  payment  accurac\'  rate  is  an  important  indicator  of  man¬ 
agement  (‘ffectiveness  and  [irogram  design.  Since  pa\'ment  accuracy  can 
lie  measiin'd  in  a  \  ariety  of  ways  tliat  portra>'  w  ide  differences  m  por- 
foi-mance.  ss.\  should  jirovide  as  full  a  disclosure  of  its  iierlormance  as 
possible  to  assist  both  management  and  the  Coiigress  in  pro\  iding  lao- 
gratn  oversight , 


The  Secretary  of  Health  and  Human  Ser\  ices  Humid  direci  the  Coiumi  - 
sioner  of  Social  Security  to  n'i>orI  all  categories  of  deii-ricd  i,'';  progi'am 
errors.  Sm  h  reporting  should  incliidi' 

case  accura<  >'  rail's  as  well  as  dollar  aci  lu'ai  y  rail"', 
all  errors  detected  during  the  sample  period,  and 
t  he  incidence  of  errors  at  t  rihiiled  to  s>  \ 

The  Secret  a  r\  should  also  di  red  I  he  Coiiimissiouer  o|  s,  u  i,il  Sim  luii  \  to 
deli'rmine 
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Agency  Comments  and 
Our  Evaluation 


I 

i 


the  extent  to  which  regional  assessment  offices  may  not  be  identifying 
errors  during  the  I’ayment  Accuracy  Study  and  the  t'ffect  such  unde¬ 
tected  erroi's  have  on  the  payment  accuracy  rate  and 
what  additional  verification  or  changes  may  be  warranted  in  the  moni¬ 
toring  of  regional  quality  review  analysts. 


In  a  letter  dated  September  2n,  1987.  the  ims  Inspector  General  said  mis 
was  in  basic  agreement  with  our  report.  (See  app.  II. )  He  added  that 
overall  procedures  on  reporting  payment  accuracy  will  be  reviewed  and. 
in  the  future,  .ssa  will  report  on  ca.se  as  well  as  dollar  accuracy. 

Concerning  our  recommendation  to  rei)ort  all  categories  of  detected  Ksi 
program  errors,  the  Inspector  General  said  s,s.\‘s  disclosure  of  payment 
accuracy  data,  both  internal  and  external,  could  be  more  uniform  and 
comprehensive.  He  noted  that  the  House  .Appropriations  Committee 
made  specific  mention  of  the  need  to  disclose  information  on  the  inci¬ 
dence  and  source  of  errors  in  its  report  (no.  l()()-25())  on  1988  appropria¬ 
tions.  (See  p.  1 1  for  a  discussion  of  the  Commit Uv's  n'port .)  The 
Inspector  General  said  that  historical  dt'finitions  and  practices  about 
payment  error  data  will  be  reexamiiu'd  to  dcti'rmine  specific  changes 
that  shoiUd  be  made.  He  said  the  ri'ports  on  paymc'nt  accuracy  covering 
the  period  ending  December  81 .  1987,  will  reHc'ct  th(>se  change's. 

Concerning  our  recommendation  to  examine  how  well  regional  asse'ss- 
ment  offices  detect  payment  errors  during  tlii'  annual  Pay  ment  .Accu¬ 
racy  .Study,  the  Insiwetor  ( ieneral  said  that  the  disparity  m  error  rate's 
we  identified  at  two  re'giems  was  not  ne'ce'ssanlN  eliie'  to  jie'i  formanee' 
deficiencies  .Ae'eeireling  tei  the  Inspe'ctor  Gi'iii'ial,  otbe'r  fae  tors,  siie  li  as 
regional  variatiems  in  e'rreir  rate's  or.  possibl>  .  de'mographie  and  e'co- 
neimic  eau.se'.s,  might  be' re'sjionsible'  for  lower  elete'ct  mn  rate's  \e'\e'rthe- 
less,  the  Inspe'cteir  General  saiel  that  tlie'ie'  is  nii’i  it  in  tlu'  thrust  of  luir 
finding  anel  mis  will  pursue'  the'  re'asons  lor  this  eiisparit\  He'  ant u  ipate's 
preliminary  results  be'lore-  1988. 

We  believe'  lllis's  plans  are'  re'spemsiN e’  to  eiiir  re'commi'ndai leuis 
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Knowing  tlu*  size  and  duration  of  {)a\  ment  errors  to  beneficiaric's  can 
supplement  payment  aeeuraiy  data.  iMiring  our  review,  we  (‘xpanded 
ssa’s  criteria  for  error,  including  errors  outstanding  before  ss.\'s  sample 
month.  V.’e  did  this  b('caus(>  it  is  possible,  as  we  noticed  during  our 
review,  that  beneficiari<>.s  can  be  correctly  paid  in  ss.x's  samjjle  month, 
yet  have  an  uncorrected  payment  error  from  periods  before  the  sample 
month.  If  an  error  was  outstanding,  we  calculated  the  net  benefit  effect 
for  each  beneficiary  as  of  the  sami)l(‘  month  ( this  effect  included  all 
uncorrected  (MTors  for  tho.se  beneficiaries  since  thi'  cast'  began):  we  then 
calculated  the  sum  of  these  payment  (-rrors  in  terms  of  monthly  pay¬ 
ments  to  beneficiaries.  .ss.\  does  not  routinely  compile  accumulated  pay¬ 
ment  errors  for  tin*  ca.ses  it  sampk's. 

Csing  an  expanded  view  of  uncorrectt'd  errors,  we  found  that  Ib.H  per¬ 
cent  (4.5  million  case's)  had  an  uncorrected  error  as  of  January  1(184  and 
that  the  ca.se.s  had  been  iti  vrror  for  an  a\'(>rage  of  about  5  years. 
Underpaid  cases  averaged  $501 .50,  which  was  worth  altout  1 .0  months 
of  1984  monthly  benefit  payme'nts  for  those'  case's,  anei  o\('rpaiei  cases 
averaged  $1,0()9.50.  whie'h  was  worth  about  2.4  months  eef  1!»84  Ite'nefits 
for  these'  e'ase's.  Whe'ii  e-eempare'ei  with  total  be'ne'fits  jeaiel  in  ('ach  e  ase', 
erreer  amemnts  ave'rage'ei  abenit  5  pe're’e'iit.  Meest  e>f  the'  e'rreer  e  ase's  wemlel 
ne)t  have  been  de'te'ele'ei  eer  e  etrre'e  te'ei  by  reetitine-  ss.\  preee  esse's.  Hven  if  an 
erreer  had  be'e'n  ele'te'eJe'ei  by  .s,s,\,  apart  from  the'  sample'  e)f  2,242  e  a.se's, 
14.0  pe'fcent  e)f  the' erreer  e  a.se's  e-eailei  neet  be'  fully  e  enae'e  te'd  bee  ause'  the'v 
were  either  teetally  eer  partially  barre'el  freem  e  eerre'e  t ieen  by  an  ss.\  aeimin- 
istrative  preeee'elure'. 


Almost  One  in  Six 
Cases  Had 
Outstanding 
Underpayments  or 
Overpayments 


Of  the  almeest  erne'  in  six  e  ase's  in  e'rreer  (  1 0.8  pere  e-nt ),  seeme'  e  a.se's  hael  all 
underpayments  e»r  all  eeve'rpayme'iits  anel  seime-  e  ase's  baei  beetle.  We>  eeim- 
puted  a  ne't  erreer  in  e;ee  h  e  ase'  anei  lounel  theet  slightly  nieire'  tb;ue  0  of  10 
erreer  ease's  had  a  teetal  ne't  unele'rjeeeyment.  Dedlai'  e-rieeis  in  e  eese's  with  ne't 
underpayments  range'el  freem  $l  pe-r  e;ise'  tee  $(i,285.80,  with  ;in  ;ive'r;ige' 
e)f  $591 .50  pe'c  e  a.se';  the*  me'ehaie  unele'i'ieayme'nt  wees  $  19.1.55  Whe'n  e  eem- 
pare'd  with  the*  tedal  leayments  meieie'  in  e'eee  h  e  eise-,  lenele'riiitiei  eleellais 
range'd  frenn  le'ssthan  1  leeree'iit  tee  245  jee'ree'nt  (the-  be'ue'fn  i;u  \  re-ee-ixe'e! 
abeeut  eeiu'-thirel  eef  the>  ameeunt  e'letitle-ei ).  with  an  a\  e'tage'  eif  4  4  pent  e'let 
In  e'eemparisem.  eleellar  erreers  in  e  ase's  with  teeteil  lee't  ei\  e'i  pa,\  ine’iits 


''Ou'  .Sei.CH.'i  See  Iiiulcrp.’i.vniceie  mi  Iim’<l  Ik'c  .lusc  .SS.\  did  n"i  iii'  lnd'  <  .1:  iumi:^  IC.  :  -I 

nioiitlily  berh’tn  t-oinpiitalion  'Ph*  U'notn  jaiA  in  1  hi'*.  >  os, •  li,i,i  h.eii  miel.  !  {m,, i  ; '.t"'  T  \  •  ,r - 
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Continued  for  an 
Average  of  5  Years 


ranged  from  less  than  $1  per  east'  to  $21.3()7.2{),“  with  an  averagt*  of 
$1,069.50  per  case;  the  median  overpayment  was  $251 .40.  In  terms  of 
total  payments  in  each  case,  overpaid  dollars  ranged  from  less  than  1 
percent  to  100  percent  (the  beneficiary  was  not  entitled  to  benefits 
received),  with  an  average  of  5.4  percent .  4'ht'  1 6.8  percent  error  rate 
included  cases  with  relatively  small  total  ('rrors.'  \V5'  comiioted  the  aver¬ 
age  without  these  small  errors  and  found  lh('  ('rror  I’ate  decreased  to 
13.1  percent. 

SSA  will  detect  and  correct  som('  errors  by  routine'  processes.  For  t'xam- 
ple,  beneficiaries  who  continue  to  work  after  retin'inent  may  be  eligible 
for  an  increase  in  their  monthly  benefits.  ss.\  jernce'diin's  require  that 
this  increase  be  made  within  21  months  after  the'  end  of  the  earnings 
year.  Cases  that  exceeded  this  21 -month  crit('rion  were  considered  erro¬ 
neous  by  .SSA  and  us,  even  though  a  futun'  roniinc'  f)roce.ss  would  eventtt- 
ally  correct  the  problem.  Of  the  ca.s('s  in  ('rror  as  of  the  sample  date.  10.8 
percent  were  either  corrected  by  the  tina*  of  .s.sa'.s  case  re\  iew  or,  Ui  our 
opinion,  were  likely  to  be  corrected  in  th('  futun'  by  oitt'  of  several  rou¬ 
tine  ssa  processes,  such  as  the  Automatic  Farnings  Heapi)raisal  Opera¬ 
tion,  intended  to  review  and  update  records  and  n'calculate  benefits. 

As  compared  with  cases  that  could  be  corri'ctc'd  by  routine  proce.sses,  an 
administrative  procedure  barrt'd  either  total  or  jiadial  correction  of  14.6 
percent  of  the  error  cases,  as  mentiont'd  t'arlic'r.  About  thn'c-fourths  of 
these  cases  had  one  or  more  errors  that  wc'ix'  totally  barn'd  from  correc¬ 
tion.  The  other  one-fourth  had  multipk'  ('rrors,  of  which  at  least  one  was 
barred  from  correction.  Ca.si's  with  iK't  ovx'i  paymi'nts  wc'ix'  more  than 
twice  as  likely  to  be  patlially  or  totally  barred  from  correction  than 
those  with  net  underpay na'Uts.  (lenerally.  when  the  underpayment  is 
not  caused  by  the  bem'ficiary,  it  can  be  (■orr('et('(l  ('V('n  aft('r  4  y('ars. 


The  number  of  mont  hs  a  beneficiai  y  recei\  ('(l  erroiu'oiis  In'iu'fits.  as  of 
.lanuary  1984.  ranged  from  a  minimum  of  1  month  to  a  maximum  of  257 
months,  with  an  average  of  61 .3  mont  hs  and  a  im'dian  of  40  months.  On 
the  average,  casi's  in  ('rror  rec('ive(i  th('  wrong  monthly  iM'iU'fit  amount 
for  70  percent  of  the  months  for  which  a  Ix'iu'fit  was  paid  or  was  due. 
Without  small  errors,  iIk*  av('rag('  was  66  p('rc('nl. 


“The  $21 .2r»7.20  ovrr]>iiynu’nt  nr»  nrnHl  1mm  atis»'  SS  \  lonih I  Moil  i  In  Im  ik  I  o  i.n  \  stuuild  hav r  had  Im'h 
i’fits  withhold  iHvausi' ol' I'hi'' ln-nrlp  iar\  had  n  . .\  .  t  [i.iid  i(  o  Mu  )i.t-''i  \  2  \f;irs 

AVV  (Icfim’d  small  rrrors  :ts  a  »  ilso  uiih  IndtAv  a  ini  ;d  .u  .  ninulah'd  ciior  ami  hr  tew  s‘,  ituoit  h 
avorago  <*rn*r.  ( ’;tsos  liad  to  moot  l»nt  h  <  rih-i  la  in  In  -  *  I'osidrrrd  so  nil 
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Conclusions 


We  compared  the  accumulated  error  totals  with  the  1984  benefit  pay¬ 
ment  to  get  another  perspective  on  the  impact  of  errors  on  beneficiaries. 
To  do  so,  we  first  had  to  express  the  accumulated  errors  in  1984  dollars 
because  inflation  had  increased  the  value  of  earlier  error  dollars.  We 
used  the  cost-of-living  increases  granted  to  Social  Security  beneficiaries 
to  increase  all  error  dollars  to  a  1984  base.  We  then  measured  the  eco¬ 
nomic  worth  of  the  accumulated  errors  as  of  the  1984  sample  date.  Fur¬ 
ther  details  on  this  constant  dollar  technique  can  be  found  in  appendix  I. 

Application  of  this  constant  dollar  technique  showed  that  the  average 
net  underpayment  per  case  of  $591.50  was  worth  $827.00  in  1984  eco¬ 
nomic  purchasing  power.  We  divided  the  escalated  error  totals  by  the 
1984  benefit  payment  for  each  case  and  averaged  the  results.  I'sing  this 
technique,  the  average  net  underpayment  was  worth  about  1,(5  months 
of  1984  benefits  for  these  beneficiaries.  The  range  varied  on  an  individ¬ 
ual  case  basis  from  le.ss  than  1  month  to  almost  17  months  worth  of 
benefits.  F'or  a  typical  elderly  couple  aged  (55  and  over  in  1984.  this 
amount  of  economic  purchasing  power  represented  about  (5  percent  of 
total  yearly  expenditures. 

Our  constant  dollar  analysis  showed  that  the  average  net  overpayment 
per  case  of  $  1 .069.50  was  woith  $  1 .342.00  in  1984  purchasing  power 
(about  2.4  months  of  1984  benefits).  The  range  was  again  wide  for  indi¬ 
vidual  cases,  from  a  low  of  less  than  1  month  to  a  high  of  almost  35 
months.  For  a  typical  elderly  couple  in  1984,  the  additional  economic 
purchasing  power  was  worth  about  10  percent  of  total  yearly 
expenditures. 


Most  uncorrected  errors  are  small  relati\  e  to  the  monthly  benefit.  These 
errors  are  not  of  the  type  likely  to  be  detected  by  routine  processes  that 
review  payments,  update  records,  and  recalculate  benefits.  Conse¬ 
quently,  these  errors  are  likely  to  continue  for  a  long  time  and  the  accu¬ 
mulated  error  amount  can  be  substantial.  The  total  lifetime  im{)act  of 
errors  on  beneficiaries  affected  by  errors  not  detected  by  routine 
processes  or  special  studies,  such  as  the  annual  Payment  Accuracy 
Study,  will  be  larger  than  the  averages  presented  in  this  chapter.  This  is 
so  because  of  the  following;  Had  the  cases  not  been  included  in  the  sam¬ 
ple,  the  payment  errors  for  most  cases  would  ha\  e  contitnied  indefi¬ 
nitely  and  the  amounts  overpaid  or  underpaid  would  ha\  e  accumulated 
further. 
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Chapter  3 

How  Errors  Affect  Beneficiaries 


SSA  does  not  currently  compile  data  from  its  Payment  Accuracy  Study  to 
disclose  the  size  and  duration  of  payment  errors  to  beneficiaries.  Period¬ 
ically  collecting  and  reporting  the  effect  of  errors  on  beneficiaries  would 
provide  an  additional  payment  accuracy  indicator,  disclosing  the  extent 
to  which  errors  are  affecting  beneficiaries’  payments. 

If  data  about  the  average  and  median  amounts  as  well  as  the  duration  of 
overpayments  and  underpayments  are  desired,  either  on  a  periodic  or  as 
requested  basis,  the  Senate  or  House  Appropriations  Committees  may 
want  to  so  advise  s.sa. 


Matter  for 
Consideration  of 
Congressional 
Committees 
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Scope  and  Methodology 


Adequacy  of  Sample 
Design  and 
Methodology 


Verification  of  Detected 
and  Reported  Errors 


\VV  wanted  to  ( 1 )  determine  if  ttu'  sami)l('  desiffn  and  methodology  of  the 
Payment  Aeeuraey  Study  could  aeeurat(*ly  measun*  errors,  (2)  compare 
1  he  errors  ss.\  finds  with  what  it  reports,  (d)  determine  whether  all 
errors  were  being  detected,  (4)  reealeulat(‘  error  rates  to  retleet  any 
d('t(‘eti‘d  errors  not  included  in  ssa's  caieulai  ion,  and  (  o)  measure  tlu' 
dollar  (‘fleet  of  (‘rrors  on  Ix'tK'fieiai'ies. 


From  a  statistical  standpoint,  we  e\alualed  tht-  sample  size  and  the  pro- 
(•('dures  us('d  for  sample  st'leetion,  ca,s('  rf'\  levs ,  and  estimating  the 
extent  of  error;  we  also  reviewi'd  the  ai>plieation  of  tin*  prt'seriht'd 
proe(‘dures. 


'I'o  verify  .ss.v  externally  n‘pojl<‘d  error  rati's,  we  obtained  jiuhlished  ss.v 
tx'ports  from  fiscal  ye;ir  IHHI  through  IPKt)  and  ss.v  repoils  to  the  House 
•Appropriations  (’ommitt(‘(“  for  fiscal  IhHC);  w('  then  comptired  data  from 
tlu'se  sources  with  th<‘  study's  supporting  data  base  We  limited  our  v(‘r- 
ifictition  of  the  d:it:i  b;ise  to  a  comparison  of  the  coding  on  most  litS4 
ca.ses  iind  the  findings  n'corded  In  I  lit-  cpialilv  a,ssunmc(‘  ri'view  folder 
.Although  a  few  diffen'uces  were  detected,  the  instances  were  not 
nutnerous  (‘tiough,  in  our  opinion,  to  afh'ct  1  lu‘  v  alidity  of  t  h(‘  data  ba.st' 
\V('  then  used  the  data  base  and  ssv's  critt'fia  for  reiiortable  ('rror  to 
recalculate  the  error  statistics  reported  internally  and  externally  hy  ss.v 
for  fiscal  vears  IPHI  -Hti 

I'o  check  th(‘  accuracy  of  the  (gialily  assiiranci'  rev  lews  done  hy  the 
regional  ass<‘s,sment  offices,  we  considered  the  rea.sonahleness  ol  the 
determinations  and  verified  the  aci  iiracv  of  the  benefit  calculation  t'or 
the  2.24'2  (  ases  sampled  in  l!tH4.  geiierallv ,  wc  used  the  data  m  the 
rev  iew  tile  W'c  did  not  alti'injit  to  mdei)endeMt  1\  verity  t In' tindings 
through  a  check  ol  t he  claims  Ibldcrs  m  the  ini s 

I'o  (  heck  if  error^s  might  be  going  nndt'le(  ted  during  ri'gional  ol  tne 
rev  lew  s,  we  aiialv  zed  A  V  ears  I  1  PS  I  S.'iot  pav  men!  ai  i  nraev  data  Siiue 
the  III  regional  assessment  ( d t ices  rev  lew e(l  (  ases  from  each  ( d  the  I'l 
IM  s.  we  (  ( Hist riK  ted  a  mai ri\  show  mg  i  I  i  the  mimher  < d  i  ases  re\  lewed 
and  ( i!  I  the  number  < d  errv h  i  a'-; ‘s  h mud  h ir  eai  h  esi  h\  eai  h  rev  lew  ing 
regional  ot  t  ue  \s  inent  loned  ea rl  lei  some  regional  ( d  t  lees  (let ect ed 
more  errors  i  ban  ot  bers  Ixs  aiise  t  bev  w ere  assigned  more  error  i  ases  tor 
rev  lew  I  n  I  irdel  t  O  e\  ablate  w  bel  her  tile  ill  I  terelli  (  S  m  error  del  ('(  1  loll 
rat es  I  ol dd  he  .ii  I  rihiit ed  solo) \  to,  b.im  e  rat  1 II  'I  I  ban  some  ot  tier  reason 
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Appendix  I 

Scope  and  Methodology 


Generation  of  Alternative 
Error  Rates 


Impact  of  Errors  c  i 
Beneficiaries 


(such  as  analysts  in  some  regions  being  better  at  detecting  errors  than 
analysts  in  other  regions),  we  used  the  logistic  (LOGIT)  statistical  model. 
We  selected  the  LOGIT  model  because  of  its  ability  to  deal  with  certain 
characteristics  of  the  sample  data,  which  we  needed  to  consider  before 
evaluating  the  statistical  significance  of  detection  rate  differences.  Spe¬ 
cifically,  the  model  (1)  permits  use  of  a  dichotomous  dependent  variable 
(an  error  is  either  detected  and  valued  at  1  or  not  detected  and  valued  at 
0),  (2)  permits  use  of  nominal  independent  variables  (regions  are  anno¬ 
tated  1  through  10  and  pscs  are  annotated  1  through  6),  (3)  allows  psc 
comparisons  even  with  small  numbers  of  sample  cases  in  some  regions, 
and  (4)  controls  for  interaction  effects,  that  is,  regions'  receiving  differ¬ 
ent  numbers  of  cases  with  errors  from  different  pscs. 


To  generate  alternative  accuracy  statistics,  including  the  errors  ssa 
deemed  “not  reportable,”  we  used  the  data  base  to  produce  case  and 
dollar  error  rates  that  included  all  detected  and  recorded  errors  for  each 
of  the  6  years.  This  procedure  was  possible  because  ssa  quality  review- 
analysts  coded  all  errors  noted  in  sample  cases  regardless  of  whether  or 
not  the  errors  were  considered  reportable.  We  used  statistical  techniques 
similar  to  those  used  by  ssa  and  projected  the  alternative  error  statistics 
to  estimate  underpaid  and  overpaid  dollars  and  numbers  of  error  ca.ses. 


To  evaluate  the  impact  of  accumulated  errors  on  beneficiaries,  we 
obtained  all  available  quality  a.ssurance  re\  i(nv  folders  for  the  1984  Pay  ¬ 
ment  Accuracy  Study  sample  from  the  10  regional  a.ssessment  offices 
responsible  for  carrying  out  the  review.  This  year  was  chosen  because 
the  data  were  the  most  current  available  when  our  review  was  started. 
Althougl  the  .January  payment  was  sampled,  not  until  the  following  fall 
were  all  cases  completed,  findings  enten'd  into  the  data  bas(',  and  the 
data  ba.se  verified.  The  1984  Payment  .Accuracy  Study  sainjile,  2.242  usi 
ca.ses  from  across  the  country,  was  rc'viewt'd  by  analysts  in  Ilu'  region  in 
which  the  sampled  beiu'ficiary  residi-d 


.SSA  considers  an  ('rror  to  ('xist  only  when  t he  sample  month  i>a>  inent  is 
incorrect.  For  the  purposi' of  measuring  the  impact  ot Crrors  on  hmieft 
ciaries,  we  includ('d  till  ca.si's  where  the  henefii  iar\  was  etther  dm' 
money  or  owed  money  Ix'caii.sc  of  an  ('i  ror,  regardh'ss  of  wlictlu'r  tht- 
sample  month  payim'Ht  was  corrt'ct  or  incorri'ct  .As  a  ttieasuri'  of  the 
impact  of  errors  on  beneficitirii's,  tlu'  amount  of  the  orror  for  a  specifit 
ca.se  was  calculated  arid  <'\ pressed  as  a  total  of  all  money  due  the  benefi¬ 
ciary  or  due  the  govt'rnment.  We  ealciilated  other  i  imiiilattve  statistics. 
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Appendix  I 

Scope  and  MethodoloKy 


such  as  total  benefits  paid,  months  paid,  and  months  incorrectly  paid; 
we  also  made  mathematical  comparisons,  such  as  the  percentage  of 
monthly  payments  in  error.  To  compare  the  impact  of  accumulated  error 
dollars  with  the  1984  monthly  benefit  payment,  we  calculated  a  new- 
total  dollar  error  (using  cost-of-living  increases  granted  to  Social  Secur¬ 
ity  beneficiaries),  expressed  in  constant  1984  dollars.  To  calculate  the 
number  of  benefit  months  represented  by  the  error  amount,  we  divided 
the  new  total  dollar  error  by  the  amount  of  the  .lanuary  1984  monthly- 
benefit  amount.  This  constant  dollar  technique  was  necessary  becau.se 
the  value  of  error  dollars  over  the  years  was  affected  by  inflation. 
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Comments  From  the  Department  of  Health  and 
Human  Services 
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nUARTMINT  or  HEALTH  A  HUMAN  SERVICES 


Otfle*  0<  Intpwtor  Ocnwd 


SEP  25  «7 


Mr.  Richard  L.  Fogel 
Assistant  Comptroller  General 
U.S.  General  Accounting  Office 
Washington,  D.C.  20548 


Dear  Mr.  Fogel: 


The  Secretary  asked  that  1  respond  to  your  request  for  the 
Department's  comments  on  your  draft  report,  "Social  Security: 
Payment  Accuracy  Rates  Are  Overstated."  The  enclosed  comments 
represent  the  tentative  position  of  the  Department  and  are 
subject  to  reevaluation  when  the  final  version  of  this  report  is 
received . 


We  appreciate  the  opportunity  to  comment  on  this  draft  report 
before  its  publication. 


Sincerely  yours, 


\  -  -  \  W  - 

Richard  P.  Kusserow 
Inspector  General 


Enclosure 
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Appendix  11 

Comments  From  the  Department  of  Health 
and  Human  Ser\'ices 


COMMENTS  OF  THE  DEPARTMENT  OF  HEALTH  AND  HUMAN  SERVICES  ON  THE 
GENERAL  ACCOUNTING  OFFICE  DRAFT  REPORT,  "SOCIAL  SECURITY- 
PAYMENT  ACCURACY  RATES  ARE  OVERSTATED" 


General  Comments 


This  report  presents  the  General  Accounting  Office's  (GAO) 
analysis  of  the  Social  Security  Administration's  ( SSA )  measure¬ 
ment  and  reporting  of  payment  error  rates  in  the  retirement  and 
survivors  insurance  (RSI)  program.  It  also  presents  data 
developed  by  the  auditors  on  the  amount  and  duration  of 
erroneous  payments  to  Social  Security  beneficiaries.  In 
essence,  the  report  concludes  that  SSA ' s  sampling  methodology 
and  procedures  for  measuring  errors  and  determining  annual 
payment  accuracy  rates  are  adeguate.  In  fact,  the  auditors 
used  the  review  data  produced  by  SSA  as  the  basis  of  all  of 
their  findings.  They  did  not  see  the  need  to  perform  an 
additional  audit  of  the  quality  assurance  review  process. 
However,  the  report  notes  that  the  data  could  be  better 
presented  to  fully  disclose  the  amount  of  error  in  the  RSI 
program,  and  to  allow  for  better  program  oversight  by  Congress 
and  SSA.  Specifically,  the  report  faults  SSA  for  not  reporting 
case  error  rates,  for  understating  the  extent  of  errors  by  not 
also  reporting  on  underpayments,  errors  from  $1  to  $5  per 
month,  and  errors  barred  to  correction,  and  for  not  reporting 
whether  errors  SSA  detects  are  caused  by  it  or  others.  The 
report  also  has  findings  related  to  the  performance  of  SSA's 
regional  assessment  offices  in  detecting  errors. 

We  are  in  basic  agreement  with  the  thrust  of  the  report  that 
more  comprehensive  disclosure  of  data  is  both  desirable  and 
feasible.  We  plan  to  review  our  overall  procedure  in  the  area 
of  reporting  payment  accuracy  data,  and  we  will  in  the  future 
report  on  case  as  well  as  dollar  accuracy  for  both  overpayments 
and  underpayments. 

GAO  Recommendation 

The  Secretary  of  Health  and  Human  Services  should  direct  the 
Commissioner  of  Social  Security  to  report  all  categories  of 
detected  RSI  program  errors.  Such  reporting  should  include: 

case  accuracy  rates  as  well  as  dollar 

accuracy  rates. 


all  errors  detected  during  the  sample 
period,  and 

the  incidence  of  errors  attribut-ed  to  3:^“. 


I 


Pam*  ;Jfi 


(iAO  HRIrSS  10  I’a>  iiiriit  Acniriicx  Rjilcs  Arc  <  )\  rrslal<'<l 


Appendix  II 

Comments  From  the  Department  of  Health 
and  Human  Serv  ices 


DecartTnent  Comments 


We  agree  that  SSA’s  disclosure  of  payment 
internally  and  externally,  could  be  more 
hensive.  The  House  Appropriations  Ccmmi*: 
(number  1  00-256  )  ma)ces  specific  mention  c 
data  that  SSA  provides,  and  indicates  th^ 
Congress  should  "disclose  the  incidence  c 
extent  of  error  attributed  to  SSA  as  wel) 
dollars  paid  erroneously."  As  we  taice  ac 
the  Comm.ittee's  instructions,  we  plan  to 
cal  definitions  and  practices  regarding  e 
towards  determining  what  specific  changes 
the  correlating  effects  on  trust  fund  ac: 
oversight  and  resource  management.  We  w: 
Committee's  instructions  beginning  with  t 
ending  December  31,  1987. 

GAO  Recommendation 


The  Secretary  should  also  direct  the  Commissioner  of  Sc 
Security  to  determine: 
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the  extent  to  which  regional  assessment 
offices  may  not  be  identifying  errors 
during  the  payment  accuracy  study  and 
the  effect  such  undetected  errors  have 
on  the  payment  accuracy  rate,  and 

what  additional  verification  or  changes 
in  the  monitoring  of  regional  quality 
review  analysts'  determinations  may  be 
warranted . 


Department  Comments 

This  recommendation  stems  from  G.= 
whether  some  regional  assessment 
others  in  detecting  errors.  Usir 
compared  the  differences  in  erroi 
possible  pairs  of  SSA's  10  regicr 
statistically  significant  differs 
rates  for  the  Chicago  and  Denver 
showed  consistently  lower  error  i 
that  the  lower  error  rates  are  di 
chance,  or  differences  in  actual 
the  report,  GAO ' s  "statistical  mr 
influences  by  mathematically  rem 
offices  receiving  more  cases  w:i; 
others.'*)  GAO  concludes  that  thi: 
two  review  offices  are  not  findir 
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Comments  Fn>m  the  IVparlmenI  of  Health 
and  Human  Sen  ires 


national  accuracy  rates  that  are  higher  than  they  should  be. 

We  do  not  agree  with  this  conclusion  that  the  disparity  in 
error  detection  rates  is  necessarily  due  to  performance 
deficiencies  in  these  two  regions.  In  our  view,  the  variances 
could  be  due  to  other  factors,  such  as  regional  variations  in 
error  rates.  Additionally,  we  do  not  believe  that  we  can  rule 
out  demographic  and  economic  causes  for  a  lower  error  rate  in 
these  regions. 

These  concerns  notwithstanding,  we  find  merit  in  the  thrust  of 
GAO'S  finding  and  will  pursue  efforts  to  determine  the  reasons 
for  these  apparent  disparities  in  regional  detection  rates.  We 
anticipate  preliminary  results  by  the  end  of  calendar  year 
1  987  . 
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